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University of Washington 
Abstract 


THE EFFECTIVENESS OF 
PARTNERING AND SOURCE SELECTION 
IN JOB ORDER CONTRACTING 


Francis S. Mulcahy 


Chairperson of the Supervisory Committee: 
Professor John E. Schaufelberger 
Department of Construction Management 
Job Order Contracting (JOC) is an alternative to the sealed bid method of procuring 
facility construction and repair services. The concept was developed by the Federal 
Government in the 1980s to reduce the processing time and administrative effort 
required for smaller construction jobs. A typical JOC contract is two to five years in. 
duration and work is issued through individual delivery orders, with cost determined by 
pre-negotiated unit prices for a myriad of tasks. However, there are various methods that 
JOC contracts are procured and administered. Due to the need for negotiations on 
individual delivery orders and the lack of a defined scope of work at the beginning of the 
contract, teamwork and frequent communication are required for a successful contract. - 
A formal Partnering process is often used in JOC to focus the parties in that direction. In 
addition, many JOC contracts employ a negotiated source selection procedure to choose 
a qualified contractor instead of using a sealed bid method. 
Research was conducted to examine the effectiveness of Partnering and source selection 
procedures to determine if their use enhances success of JOC. Thirty-five sites 
nationwide were surveyed to assess opinions on performance of the contract from both 
the owner and contractor perspectives. Areas for analysis included construction 
performance, administrative support, the owner-contractor relationship, and the 
participants’ satisfaction with the contract. Analysis of survey data revealed that owners 
and contractors perceived advantages in the use of Partnering and source selection 


procedures in improving key areas of contract performance and overall] satisfaction. 
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Introduction 


Job Order Contracting (JOC) is a relatively new method for facility owners to 
contract for construction services. Developed by the Army as an alternative to the 
traditional design-bid-build method of contracting, JOC has been very successful in 
expediting delivery, reducing in-house workload, and allowing owners flexibility, but 
there are problem areas. Awarding JOC contracts based on low bid vice negotiated 
source selection may create situations where negotiated unit prices inaccurately reflect 
the true cost of construction, leading to adversarial relationships and poor performance. 
In addition, the nature of the contract lends itself to a Partnered approach, where all 
stakeholders of a project can trust each other and work together to achieve the best 
results. Non-Partnered JOC contracts may tend toward more adversarial relationships 


and inhibit contract performance. 


This thesis attempts to study the effectiveness of Partnering and source selection 
as they are employed in the administration of Job Order Contracts nationwide. In 
Chapter I, the Literature Review discusses both traditional sealed bid and JOC 
contracting methods in addition to a Partnering and “Best Value” source selection 
procedures. The Research Methodology in Chapter II explains which areas of JOC 
performance were analyzed, how sites were selected for survey, the development of the 
surveys, and the collection of data. The survey response data and the results of the 
statistical analysis are presented in Chapter III, Analysis of Data. Finally, Chapter [V 
contains the conclusions and recommendations of the study. The surveys used in the 
collection of data are included in Appendices A and B, while Appendix C summarizes 


the comments received in the surveys. 


Chapter I: Literature Review 


1.1 Traditional Sealed Bid Public Works Contracting 


Public works contracts vary from private sector contracts in that legislation sets 
the contracting guidelines for contracts and the public interest demands safeguarding 
process integrity. Thus, the requirement for full and open competition is a hallmark of 
public contracts. To minimize the chance of favoritism or corruption, the sealed bid 
contracting method was developed to award a contract on the basis of price alone. Also 
referred to as the “Design-Bid-Build” process, this system has long been the most 


prevalent option for large and small public works contracts. 


The first step in the procedure is to prepare a design and may be done with in- 
house personnel or through a contract for Architect/Engineer (A/E) services. Generally, 
smaller projects are designed in-house, while A/E designs are used for larger projects 
due to the higher complexity and workload. Once the design is completed and reviewed, 
a bid specification is prepared, then sent out to contractors for preparation of bids. The 
bid specification defines the rules for that particular procurement, including the format of 


the bid, any administrative requirements, and the deadline for bid submission. 


Contractors estimate the cost of the project based on the design and submit a 
lump sum price for a project that remains secret to everyone involved in the process until 
a predetermined time, known as the “bid opening.” At bid opening, the bids are 
compared in a public forum, then the contract is awarded to the bidder who has met all 
requirements of the bid specification, is capable of performing the work, and will charge 
the lowest sum to complete the project. The sum can either be measured as one price for 
the entire project, several prices for different parts of the project, unit prices for elements 


of the project based on estimated quantities, or a combination. 


The main strengths of sealed bid contracting are its simplicity and objectivity. 
Through the period up to groundbreaking, there is a relatively easy recipe to follow: 
design the project, develop a bid specification, advertise the project, collect and open 
bids, then award the contract to the low bidder. The selection criteria are objective and 
clear; from the pool of bidders who properly and fully respond to the solicitation and are 
deemed responsible to perform the work, the lowest price wins the job. Since no 
judgement is required in the determination of the winning bid, it is difficult to corrupt the 


system with bribery, favoritism, conflicts of interest, or other unseemly methods. 


The most significant benefit achieved through the sealed bid method is the high 
degree of competition that is encouraged. There is no limit to the number of bids that 
may be received in a standard sealed bid procurement, so control of the initial cost of 
construction is excellent. It also allows every responsible contractor in the community 
the opportunity to receive work from the public owner, which is an important social goal 


in most government entities. 


1.2 Issues With Sealed Bid Contracting 


Problems Developed Over Time 


Sealed bid contracts have been used in public works construction for a very long 
time. There have been many developments in both contracts and construction that have 
changed the substance of sealed bid contracting. The technical complexity of projects has 
grown immensely, resulting in the specialization of subcontractors. The lessons of 
construction litigation have been incorporated into contract documents, while litigation 
itself has expanded its role in the resolution of disputes.' Contractors have become 


sophisticated experts on contract interpretation in an effort to protect their interests and 


' Michael T. Kubal, Engineered Quality in Construction: Partnering and TQM (New York: McGraw-Hill, 1994), 
163. 


s( 


maximize profit.” Increasing attention has focused on the problem areas within sealed bid 


construction contracting. 


Linear Timeline 


A problem inherent in the sealed bid method is the long lead time required for 
completion of a given project. The cost for a simple “one-follows-another” process is an 
inefficiency in use of time.’ The private sector, realizing that time is money, has always 
recognized this a major cost and has increasingly abandoned sealed bid contracts. 
Traditionally, public works contracting officials have never placed a high emphasis on 
speed; However, current pressure to make government more efficient has changed this 
paradigm.* Though design times will vary based on scope, the procurement process 
times for large and small projects can be similar, which makes the process very inefficient 


for smaller work.° 4 


H | 


Administrative Burden on Small Projects 


The administrative costs of sealed bid procurement must be considered in 
considering the efficiency of the process, especially when compared to the value of the 
construction services acquired. Design contract administration, bid specification 
preparation, advertisement, and contract award expenses may seem relatively minor 


when compared to a $14 million contract for a new administration building, but can be 


? Andrew M. Civitello, Contractor 's Guide to Change Orders (Englewood Cliffs, NJ: Prentice-Hall, 1988), 5. 
+David S. Haviland, Project Delivery Approaches: An AIA Guide (American Institute of Architects, 1975), 6. 


‘William B. Moore and Carl F. Stout, Job Order Contracting: A Procurement Success Story (Bethesda, MD: 
Logistics Management Institute, 1988), 2-1. 


>Dean T.Kashiwagi, Job Order Contracting Performance - 1998 (Arizona State University: Center for Job Order ( 
Contracting Excellence, 1998), 2. . 


alarmingly large when applied to a $125,000 remodeling project.° In most public 
contracting branches, smaller projects significantly outnumber the larger dollar value 


work, which greatly magnifies the inefficiencies’ total effect. 


Adversarial Relationships 


The roles and responsibilities of the various parties have divergent motivations in 
sealed bid contracts. The three-legged stool of owner, designer, and builder is dependent 
on each party fulfilling their separate responsibility to achieve a successful project. 
However, each party’s motivation is to reduce the size of their leg, which has the effect 
of shortening the stool, or in this case, limiting the success of the project. In comparing 
the roles of the parties once design has been completed, it 1s easy to see how adversarial 


positions are readily generated throughout the project.’ 


The owner’s role in the project, once requirements are identified, is to contribute 
resources in the form of project funding. The sealed bid method ensures that the smallest 
possible initial contribution will be made. Because of the assumption that the design is 
complete and free of defects, the owner views change orders as an attempt by the 
contractor to increase profit and will therefore strive to minimize funding for change 
orders.* If the contractor indeed proves that the design is flawed, the owner will look to 
the architect for no-cost redesign or for compensation under design liability if 
construction costs are incurred due to the flaw. Either way, additional funding for design 
flaws will be resisted as well. The result is that the owner will focus on minimizing 


contributions to the project. 


With the design complete, the architect/engineer has already invested the majority 


of effort, and now shifts to the role of design interpreter and advisor during the 


°K ashiwagi, 17. 
: 
Kubal, 47. 


’Civitello, 25. 
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construction of the project. However, since the fee for services has already been set, the 
firm will look to minimize the expenditure of additional effort, as that will take away 
from the existing profit.” Often, this results in a tendency to interpret design 
inconsistencies in a manner that results in the least additional effort. In admitting a design 
error, the responsibility falls on the architect to provide redesign at no cost. As that 
results in added expenditures, the architect will seek fault elsewhere, as in the owner’s 
miscommunication of intent or the contractor’s poor workmanship. Like the owner, the 
architect’s contributions to project success will always be constrained by a divergent 


motivation.!° 


The contractor in sealed bidding acquires the overwhelming majority of the risk 
during construction, and consequently has the most to lose if the project fails. 
Conversely, the success of the project can be measured in profit earned from the job. 
Since there is a natural conflict between profit and costs throughout the project, the 
contractor’s motivation is to minimize costs, and consequently, his or her contribution to 
the project.'! The pride in workmanship displayed by the skilled craftsman takes a 
backseat to the fiscal reality of meeting a minimum standard. In some cases, profit as bid 
is minimal even when costs are in line with estimates, and the contractor’s interest in the 
project is minimal as well, diminishing the success of the job.'? In other cases, the 
contractor’s costs are above estimates and the project 1s losing money, tempting the 
contractor to reduce costs below the level that would fully satisfy the specifications. 
Ultimately, this divergent motivation has the potential to significantly impact the success 


of the contract. 


*Kubal, 50. 


HH Murray Hohns, Preventing and Solving Construction Contract Disputes (New York: Van Nostrand Reinhold, 
1979), 23. 


"ubal, 74. 


2+ Tohns, 7. 
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The separate responsibilities and conflicting motivations mentioned above create 
a situation that breeds mistrust and conflict between the parties. Clearly, the most 
significant conflict is between the owner and contractor, due to the influence that the 
contractor’s performance has on the outcome, the importance of the outcome to the 


owner, and the proportioning of risk between owner and contractor. 


In many disputes, the owner perceives that the contractor’s goal 1s to recover 
unanticipated costs, increase profits, or improperly shift risk to the owner. On the other 
hand, the contractor may feel that the owner ts trying to get “something for nothing” in a 
change order or improperly shifting risk to the contractor. This adversarial position 
inhibits communication, as each side guards information that would give themselves an 
advantage in negotiation. The hard feelings generated in a dispute will spill over into 
other areas of the work, as resentment leads to poor communication and problems in 
otherwise satisfactory parts of the project. In the end, the attainment of success becomes 


impossible. 


This situation becomes aggravated when the successful low bidder has 
intentionally priced the work at lower than actual cost to “buy in” to the job. The 
contractor starts at a disadvantage, and must either seek to recoup money “left on the 


table” through inflated change order prices or by installing substandard work.” 


Defective Plans and Specifications Lead to Unpredictable Cost 

The sealed bid system assumes that the contractor can give one lump sum price 
based on a complete set of plans and specifications. If the plans and specifications 
contained every piece of information the contractor needed to perform the work and 
were free of errors, then the price as bid should be the price for the complete project. In 


practice, this rarely occurs.'* The design package usually contains mistakes, conflicts 


3 Hohns, 21. 


MHohns, 34. 


“| 
between different parts of the design package, or omissions of necessary details. At best, 
these details are discovered in time for changes to be made at no cost to the owner. At 


worst, work that has already been constructed must be torn out at tremendous cost. 


Most design driven changes, however, will cost the owner extra money and may 
extend the completion date of the project. The sealed bid has been called a “guaranteed 
minimum price,” since the final cost will not be known until after construction is 
completed and all potential contractor claims are resolved."’ This is especially 
problematic in public works projects, as a fixed budget allowance permits little, if any, 


change in the total contract price. 


Limited Value Engineering 


The sealed bid method only involves the contractor after the design is complete, 
so there is no potential for the contractor to offer valuable advice and assistance in ( 
development of the design. From methods to materials, the contractor often has a 
realistic account of the constructability, cost elements, and market conditions that will 
affect the project. The designer, though qualified in the art and technology of the 
construction, lacks the contractor’s perspective and makes many assumptions during the 
design process that could easily be clarified through collaboration with the contractor.'® 
Though there is opportunity for value engineering after the contractor reviews the 
design, this is limited by the design as a whole and usually results in little more than 


material substitutions that save only a small percentage of the overall cost. 


David E. Allen, Changing Delivery Methods: Is Lump Sum Bidding Dead? (Presentation to University of 
Washington CM520 Class, November 18, 1999. 





16 Taviland, 9. 


Construction Firms Treated as Commodities 


The major assumption in the sealed bid process, and possibly the most flawed, is 
that all bidders will perform at the same level. Rather than recognizing construction 
management as a professional service, it is viewed as a commodity that can be purchased 
like lumber or concrete.’” Essentially, performance is treated as a constant and price as a 


variable in the sealed bid equation. 


In reality, each contractor is different, with various levels of qualification, 
experience, training, organization, and competence. Each contractor will approach the 
job differently, with alternative methods, equipment, and schedules. Some contractors 
have developed strong relationships with subcontractors and suppliers that will serve the 
project well, while others have reputations for late payment and unethical practices in 
their dealings with subs and suppliers, setting the stage for work disruptions, material 


delays, and claims. 


Declining Construction Industry Performance 


Though the issues mentioned above have a negative effect on a specific project’s 
success, the impact of many years of sealed bid contracting practices has had an adverse 
impact on the construction industry as a whole."® To be a top performing firm in any 
industry, a percentage of the profits must be reinvested in research and development, 
training, and retention of high quality personnel. Sealed bidding forces construction 
profit margins to the lowest possible level, severely curtailing the reinvestment ability of 
the construction industry. Also, there is no reward for outstanding performance in the 
sealed bid system. Superior performance takes extra effort, which drives costs higher, 
making bids higher and less competitive. Performance to the minimum acceptable quality 


level reduces cost and makes bids more competitive. This results in a reverse incentive 


'7 allen. 


18 ashiwagi, 5. 
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for excellence, a sure sign of an unstable industry.” 


Dr. Dean Kashiwagi has created a model that shows the relationship between 


competition and performance as it relates to the construction industry (Table 1). 


Table 1: Construction Performance Model”° 


High Performance High Performance 


Perceived High Cost Competitive Cost 


I 


Minimum Quality 


Low Cost 


Declining Performance 





Quadrant I represents the effect of lump-sum low-bid contracting. 


Competition reduces cost, but reliance on low price drives down performance. 


The system has also created an attitude of gamesmanship in many low-bid 
contractors. Those contractors who have been in the sealed bid business for a few years 
have often learned the rules of the sealed bid game “the hard way” after losing money on 
previous contracts.*' The scenario becomes a “win-lose” situation, where the owner 
becomes the opponent and “tactics” are employed to maximize profit. Although there is 


high competition for the initial bid, contractors recognize that once they are awarded the 


2 Kashiwagi, 5-7. 


20k ashiwagi, 6. ( | 


I Civitello, 5. 
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job, they are in a sole source negotiating arena concerning change orders.” Instead of 
waiting for the discovery of change orders, contractors “prospect” for errors, omissions, 
or conflicts in the design.” After assessing the potential value of changes, strategies for 
resolving the changes are developed to maximize the contractor’s advantage during 


negotiation. 


13 Job Order Contracting 


Background 


Job Order Contracting (JOC) was developed by the Army in Europe as an 
alternative to the traditional sealed bid method of construction contracting.** At the time, 
there was tremendous pressure on the Army contracting officials to reduce the timeframe 
needed to procure construction services, to make progress on a backlog of work orders 
despite a shrinking in-house staff, and to improve quality. JOC was conceived as a way 
of rolling many small jobs into one large contract awarded to a single provider using a 
unit price format. The main goal was to minimize the administrative contracting burden 
by eliminating the need to develop separate bid specifications and competitively bid each 
single job. The large contract’s general requirements could be negotiated once, scaling 
back managerial requirements to just scope definition and quality issues. In addition, it 
was hoped that dealing with one contractor would significantly reduce the in-house 


effort to supervise a myriad of small contracts with many different contractors. 


22Hfohns, 9. 
3 Civitello, 87. 


ay cif T. Erickson and Patrick D. Murphy, “The Job Order Contracting Solution”, Civil Engineering (April 1994), 
70. 
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The JOC Concept 


A Job Order Contract (JOC) is an indefinite delivery-indefinite quantity contract 
that usually is awarded for one year with up to four option years. Specific projects are 
not defined at time of award. Instead, the contract encompasses a broad selection of 
facilities construction, maintenance, repair, and renovation type work that may develop 
during the duration of the contract. Unit prices determine the overall project cost, but 
JOCs vary from traditional unit price contracts 1n that the basis for pricing is a pre- 
published unit price book that covers costs for thousands of individual construction 
tasks. The price book is part of the contract specifications and can be either a 
government-produced book or a private sector estimating guide, such as the Means 
Estimating System.” The contractor’s bid is actually a coefficient applied to the book’s 
unit prices to decide the price of work. The coefficient covers the contract’s general 
requirements, taxes, labor burden, overhead, and profit. For example, a coefficient bid of 
1.06 would increase the as-published cost of line items by 6% to cover all costs, ( | 
overhead, and profit of the work, with no other markup required. For example, a line | 
item for installing nylon carpet with a list cost of $20.84 per square yard would be priced 
at $22.09 per square yard under this contract. 


The process for issuing work under the contract is relatively simple, as the 
contractor has already been selected and has satisfied all of the general requirements such 
as bonding, safety plans, quality control plans, and insurance. The level of design may 
vary greatly from a site visit with the requesting party to verbally communicate the work 
requirement to a fully designed package that was prepared for sealed bid. Once the 
contractor and owner agree on a scope of work, a detailed cost estimate is submitted by 
the contractor listing all unit prices required for the job. After the owner verifies that the 


line items and quantities listed are proper, then a job order is issued and construction 


?>Erickson, 69. 
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begins. The process from identification to construction ranges between 20 - 30 days.” 


Advantages of JOC 


The JOC concept has several key advantages over the traditional design-bid-build 
method that have impressed facility owners in both the public and private sectors. 
Though the most significant benefits are attained through accomplishment of small repair 


and renovation jobs, the advantages apply to large JOC construction projects as well. 


Response Time 


The most recognized benefit of JOC is by no coincidence the reason it was 
created in the first place: to shorten the response time for customers requesting 
construction services. Since the JOC contractor has already been selected, the time 
required for bid specification development, advertisement, solicitation, and contractor 
award are eliminated for individual projects.”’ Since the JOC process does not require a 
full design, a site visit is all that is needed to convey information to the contractor. For 
very small jobs that are simple in scope, this usually means that a design can be bypassed 
completely, with the contractor providing details on how the work as scoped should be 
accomplished. The owner, rather than creating the design, simply reviews and either 
modifies or approves the proposal.”* A job order can be issued to the contractor in less 
than 30 days of a customer request, with a comparable project taking up to 180 days to 
award a contract under the design-bid-build process.””? A comparison of Army 
procurement times between JOC and non-JOC projects showed that JOC delivery orders 


26x ashiwagi, 19. 
27x ashiwagi, 20. 
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are issued in 18-27% of the time it takes for non-JOC awards.” 


Reduced Administrative Burden 


With the elimination of boilerplate requirements from individual jobs, the 


administrative burden of JOC construction is minuscule compared with design-bid-build. 


Paperwork is reduced to only those documents pertaining to the technical aspects of the 
work. The information generated during site visits, scope definitions, discussions on cost 
estimates, and schedule delineation is recorded in a streamlined manner and contained in 
a file for later reference. Though quality assurance and supervision are still required, the 
ease of working with one contractor as compared to interfacing with small individual 
contracts is invaluable. For instance, the procedures for accomplishing work at a 
particular site need only be explained once, unlike separate bids where individual 
preconstruction meetings and training are required for each job. In addition, the owner 
and contractor personnel that interface regularly become familiar with each others 
processes, minimizing confusion and promoting efficiency. Because the contracting staff 
spends less time on individual procurements, they are freed up to process a higher 


number of work requests, which allows for a rapid reduction in maintenance backlog.*! 


Lower Design Costs 


As stated above, design costs can be practically eliminated for smaller work 
items. Though an owner-provided design may be necessary for larger and more complex 
JOC projects, design level of effort may be reduced to provide only those details that are 
outside the contractor’s area of expertise. This “shared” design effort means that for a 


building electrical upgrade, the designer would focus on the building’s load and required 


*OMoore, 2-2. 
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panel and transformer size, while the contractor could determine conduit routes and 
receptacle placement. Since most JOC contract specifications include a requirement that 
all work meets applicable codes, many typical bid specifications and submittals can be 
omitted. In this manner, effort of the Architect/Engineer is reduced and design costs are 


minimized.*” 


Value Engineering and Constructability 


In those cases where design is required, the JOC contractor can participate in the 
preparation of the design and advising the designer on constructability issues. In the 
electrical upgrade example cited above, the contractor may not only reduce the design 
effort, but reduce the construction cost as well. By teaming with the designer to optimize 
placement of the transformer and electrical distribution panels, the amount of structural 
demolition and length of cable runs can be reduced. In a traditional design, the designer 
applies his strength, engineering knowledge, along with a weakness in constructability 
experience. The inclusion of the contractor as part of the design team eliminates the 
weakness, creating a much more complete and cost-effective design.”’ This also allows 
for a shorter construction time, as the contractor is intimately familiar with the design 


and can influence the design to allow for speed of construction. 


Teamwork 


The traditional sealed method promotes adversarial relationships, yet the JOC 
format lends itself to an environment of teamwork and cooperation. The long-term 
nature of a JOC encourages both contractor and owner personnel to work together to 


achieve a mutually beneficial relationship. 


32 Erickson, 69, 
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For the contractor, the incentive is future profit. Since the price is already set by 
the coefficient, the contractor can only increase profit in an individual job order by trying 
to negotiate unreasonable line items and quantities or by reducing quality of 
workmanship. If negotiations become difficult and quality suffers, the owner will choose 
another contracting avenue and few job orders will be issued to that contractor. On the 
other hand, if negotiations are straightforward and honest, and high quality workmanship 
is the norm, then the JOC contractor can expect many more delivery orders.** The 
mathematics are simple: small profit on many projects is greater than a large profit on a 
handful of jobs. The natural incentive, then, is for good performance at reasonable prices 


and a pleasant experience for the owner. 


The owner’s representatives also have a vested interest in the success of the 
contract. For those sites with unpleasant experiences in sealed bid contracts, a large 
backlog of work requests, and impatient customers, a JOC can be an exceptional way to 
accomplish work compared with other methods available. In fact, 100% of Army \ . 
Installation Commanders surveyed in 1988 declared JOC a “must-have” contracting 
tool.*° Like the parable of “The Goose that Laid the Golden Egg,” the last thing the 
owner wants to do is kill the goose. Though not abandoning the requirement to 
safeguard their interest, the understanding that an adversarial relationship can eliminate a 
viable contracting option encourages owners to treat the contractor with the respect due 
a valued business partner. Sometimes, this can be a difficult transition for those who are 
ingrained with the “cops and robbers” mentality of sealed bid contracts, but many 
officials welcome the chance to relinquish hard line stances and conduct business in an 


environment of respect and open communication. 


+4E rickson, 70. 
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Enhanced Quality 


Free from focusing on administrative requirements and adversarial positions, the 
owner and contractor personnel can turn to the important elements of construction: time, 
quality, and customer satisfaction. The teamwork that exists between contractor and 
owner allows the tenant to be intimately involved in all phases of the job, from design 
requirements to scheduling outages and work disruptions.*” The job can be scheduled 
better since tenant and contractor needs can be meshed through open discussion, and 
contractor work stoppage can be minimized, keeping costs down. The natural incentive 
for a quality product and a satisfied customer, if fostered by all parties, will result in a 


successful project.” 


Minimal Change Order Rate 


JOCs typically enjoy a very low change order rate due to the integration of the 
contractor in scope definition and design, the pre-negotiated unit price system, and the 
shift away from adversarial relationships.” The changes that do occur are mainly 
customer requested, due to the pre-priced nature of modifications and ease of execution. 
The joint scoping process and open communication during construction generates a 


valuable flow of ideas that obviate changes, resulting in a reliable up-front cost. 


Because the claims process is inherently adversarial, its use is avoided in JOC. 
Both contractor and owner work to resolve any differences at a lower level, realizing 
that a claim on one delivery order can inhibit the performance and/or award of future 


delivery orders. 


37™MAoore, 2-6. 
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1.4 Partnering 


The Partnering Concept 


The inherently adversarial positions in sealed bid contracting, with each side 
maintaining divergent motivations and general mistrust of the other, place barriers 
between the parties when a disagreement arises. The atmosphere of mistrust, combined 
with the litigious bias of modern construction disputes, results in carefully screened 
communication and a “win-lose” mentality.” True resolution of issues cannot occur in 
such an environment. Formal Partnering seeks to avoid disputes through frequent and 


open communication based on a relationship of trust.*! 


Partnering was developed to prevent barriers from springing up in public works 
projects by keeping the parties committed to a common vision and fostering trust among 
participants. The common vision is determined by a consensus of the group, and usually ( 
focuses on goals for project success, including customer satisfaction, quality, time, and 
dispute resolution. Trust leads to frequent and open dialogue, which is essential in 
resolving disputes. It is through participants working together to achieve a common 
vision, with a foundation built on strong communications and trust, that successful 


projects are realized.” 


The successes of Partnering are well documented. In recent years, the use of 
Partnering on construction projects has seen a dramatic increase.” Its cornerstone is a 


partnering agreement where both parties focus on a common set of project objectives 


“Thomas R. Wame, Partnering for Success (New York, NY: ASCE Press, 1994), }. 
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and problem solving methods.” It is best applied in large, long-term projects where the 
relationship can be developed and there is an incentive to work together.” The Army 
Corps of Engineers pioneered use of Partnering in the 1980s and has seen a dramatic 
decrease in its volume of contract claims and appeals. From 1988 to 1994, claims were 
down 71%, while 365 appeals were filed in 1994, down from 742 appeals in 1991.” The 
Air Force used Partnering on a $226 million Large Rocket Test Facility, finishing 114 
days ahead of schedule and $12 million under budget without any claims or appeals.*’ 
The Navy has implemented Partnering on its large Base Operating Support Contracts 


and Job Order Contracts, as well as some individual projects. 


Implementation of Partnering in JOC 


The format of the JOC contract encourages teamwork through the long-term 

relationship and inherent incentives that exist for each stakeholder.** The implementation 
of a Partnering program formalizes these incentives into a tangible agreement and is a 
natural extension of the JOC concept. The first step in the process is to agree to Partner, 
which should be done as soon as possible after contract award. Each party should select 
a Partnering “champion,” a member of management who will be on a site for the 
duration of the contract and will ensure that the Partnering environment is promoted and 


maintained.”” 


4 Wame, 13. 

Kubal, 129. 

46 4 dministrative Conference of the United States (ACUS), Toward Improved Agency Dispute Resolution: 
Implementing the Administrative Dispute Resolution Act (February 1995) Ch 3, Sec II. 
<www.adr.af.mil/afadr/library/docs/acusrpt.html> 


“TUS. Air F orce, Memorandum Advances in Alternative Dispute Resolution (ADR). 
<www.adr.af.mil/afadr/library/docs/secrep2.html> 
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An initial Partnering session should then be scheduled before commencement of 
work under the contract. During this session, which may last one or more days, the 
stakeholders become familiar with one another and the overall objectives of Partnering. 
The next step is to develop a charter that will outline the mutual commitment to success, 
how that success will be measured, what tasks must be completed to achieve success, 
and a methodology of how to resolve disputes on the way to success. Finally, the parties 
must carry this charter with them into the project and treat it as a living document. 
Successful Partnering takes a daily commitment to its principle and the objectives set 
forth in the contract. Follow-on sessions should be scheduled to perform “routine 


maintenance” and to ensure that the charter is being adhered to.” 


15 “Best Value” Source Selection Procedures 


The Best Value Concept 


“Best Value” Source Selection is a term coined in the Federal government for a 
contractor selection process developed as an alternative to the sealed bid method. It is 
based on the common sense premise that price alone may not be an acceptable criterion 
in purchasing products or services. Rather than selecting the lowest price, an owner 
applying source selection procedures determines the criteria that 1s important for the 
project, and invites contractors to submit proposals that will be measured against that 
criteria for final selection. The process allows for negotiations between the owner and 
interested parties, so that all necessary information may be gathered before a selection is 


made. 


The criteria vary for each procurement, but price 1s always considered to some 


degree. Common evaluation factors include past performance, key personnel 





OWame, 36. 


21 


qualifications, operational methods, and organization plans. Project specific criteria, such 
as schedule, value engineering proposals, or design considerations are also standard. 
Since the relative importance of a given factor will vary between projects, each criterion 


is assigned a weighted value in the source selection plan.”' 


After negotiations have been conducted, the contractor proposals are reviewed 
and graded by an evaluation board consisting of a group of officials specifically 
appointed for the project. The proposals are then ranked based on their weighted grades, 
then compared with their price to determine which proposal represents the best overall 
value to the owner. Typically, the owner is willing to award the contract to other than 
the low-priced offerer since a modest increase in price may garner a higher quality 


product with lower risk of nonperformance or failure. 


Source Selection Advantages and Disadvantages 


The biggest advantage to source selection is the ability to apply business 
judgement to procurement. By selecting critena that are key to the procurement and 
choosing a contractor based on the criteria, the owner is “steering” the procurement in a 
positive direction, vice the random chance element of sealed bid.” In avoiding the sealed 
bid practice of buying into the job, the source selection process allows the owner to pay 
the “night price” initially, negating the need for the contractor to make profit up through 
later modifications.” This reduces adversarial tensions and makes for smoother contract 
administration, with the added benefit of more predictable costs. By examining the 
contractor’s past performance and qualifications of key personnel, those contractors that 


have the strongest resume and present the least risk of poor performance would stand a 


3 Best Value Buys.” Government Executive Magazine (online), January 1995. 
<www.govexec.com/procure/articles/0195prs6.htm> 


«Best Value Buys”. 


>> cherie Winston, “Pentagon Pumps Up Performance.” Engineering News Review (online) (October 1999), 
<www.enr.com/news/enrb1100.asp> 
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higher chance of selection.” 


The negotiation process also promotes communication between the owner and 
potential contractors, which leads to clarification of scope and technical assumptions 
prior to contract award.” Unlike sealed bidding, the proposal can explain the price by 
listing the assumptions that the estimate was based on. If contingencies were included 
due to uncertainties in the project, they can be clarified by the owner during negotiations 
with a reduction in cost. Likewise, if the cost as proposed left out necessary items, the 
contractor can add those in the initial price, saving a potential change order down the 


line. 


The key disadvantage is the additional administrative effort required to conduct 
the source selection process compared with the simple sealed bid process. Though 
selection procedures may be streamlined to fit different scale procurements, they still 
consume valuable resources like time and personnel. Also, some question why the public A 
owner should pay anything above the lowest price offered, and because the use of Ni | 


subjective judgement is involved, bid protests and process integrity is of concern.”° 
1.6 Summary 


The Job Order Contracting format promises the facility owner a cost-effective 
way to expedite the accomplishment of facility construction and repair projects with a 
higher quality and reduced administrative effort. Though the requirement for a design 
may vary by project, the contractor should always be integrated into design development 


as early as possible to ensure the best result. 


$4.45 
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Selection of a JOC contractor should be one through a source selection 
procedure that emphasizes the best value to the owner vice the lowest price. Because of 
the long-term nature of the relationship, it is important to apply business judgement in 
the choice of an important partnership and the pitfalls of sealed bid contracting should be 
avoided. Once selected, the partnership should be formalized through a Partnering 


agreement to define the common vision and guiding principle of the participants. 


By using the JOC format, selection of a quality contractor, and committing to a 
shared vision based on open communication and trust, public facility owners can 
revolutionize the way they accompljish construction and repair projects, increasing 


customer satisfaction and reducing staff workload in the process. 
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Chapter II: Research Methodology 


To study the effectiveness of Partnering and source selection in the 
administration of JOC, a survey method was chosen to evaluate the opinions of both 
owner and contractor project managers on the performance of their contract. The survey 
was designed to measure a broad range of issues, from direct construction of the project 


to the elements forming the contractual relationship between the parties. 


The survey responses would be segregated into four different populations for 


comparison between each other: 


1. Partnered contracts procured through a negotiated source selection using a Request 


For Proposals (RFP) 


2. Non-Partnered contracts procured through a negotiated source selection a RFP ( 


3. Partnered contracts procured through competitive bidding (low-bid) 
4. Non-Partnered contracts procured through competitive bidding (low-bid) 


The anticipated results should show that Partnered contracts where the source 
selection process is used have the highest performance levels of all four populations, 
while non-Partnered/Low Bid contracts should have the lowest levels of performance. 
The other two sample populations’ performance levels should fall somewhere in 


between. 


Types of Contracts Studied 


Job Order Contracting had its start with the U.S. Army, but the concept quickly 
spread to the other Armed Services, then throughout the Federal Government. Today 
many state universities employ the JOC concept as well. However, though the concept 
remained the same, the names that were given to the contract format changed. The Army 


and Navy still refer to the concept as JOC, but the Air Force has termed it “Simplified 
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Acquisition of Base Engineer Requirements” or SABER. The Coast Guard calls it an 
IDIQ contract, for Indefinite Delivery, Indefinite Quantity. The various universities that 


were studied had various names for the JOC process, but the concept remained intact. 


Therefore, the study was limited to JOC, SABER, Coast Guard IDIQ, and 


shalhtee delivery order type contracts that met the following criteria: 

1. Work issued through individual delivery orders under an umbrella contract 
2. Line item pricing based on a unit price book with contractor-bid coefficient 
3. Multi-trade discipline capability 

4. Potential long term contractual relationship (2-5 years) 


Contracts eligible for study included only those that were ongoing or were in the 
closing stages of completion to ensure project managers were very familiar with 
performance. Contracts that had not been in place long enough for a track record of 


performance were not included. 


Areas for Analysis 


Based on the research conducted in the literature review, four key elements of 
the JOC process were identified as important topics for data collection. These elements 
consist of the performance of construction under delivery orders, the administrative 
support required for construction, the quality of relationship between the parties, and the 


overall level of satisfaction with the contract. 


The performance of actual construction manifests itself in much the same manner 
as traditional construction. The three basic concerns for a construction project that 
would be measured in the survey are quality, on-time completion, and safety. In addition, 
job order contracting is heavily reliant on subcontracting, so the scheduling and 
performance of subcontractors are of major importance for a successful JOC contract. 


Also, warranty service is an area of concern with so many small projects issued under the 
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contract and needs to be addressed. 


The JOC concept is mainly celebrated for its streamlined nature and high level of 
responsiveness to an owner’s needs. Since many projects issued under a JOC are of an 
urgent nature and must be accomplished in the most efficient manner possible, the 
success of this type of contract heavily relies upon the support element. Therefore, 
information needs to be gathered on all areas of administrative support for JOC 
construction. Much of this support must work both ways for a contract to be successful, 
so most of the data was gathered from both the owner and contractor viewpoints. Key 


areas of support included: 
* Responsiveness and timeliness of administrative support (requests-for-information 
(RFI), submittals, reports, etc.) 
* Innovation and Value Engineering (contractor’s generation of, owner’s reception to) 
* Contractor’s responsiveness to special needs of the clients ( q 
* Ability to solve and prevent scheduling and site coordination problems 
* Contractor’s management effectiveness (ability to handle many simultaneous orders) 
* Level of administrative effort required of both owner and contractor 
* Reasonableness of the owner’s inspection or Quality Assurance program 


The foundation of success in both the construction and support elements is the 
underlying relationship that exists between the contractor, owner, and in most cases, the 
end-user. Partnering was designed to apply a formal structure to this foundation and 
strengthen the connections that are necessary for a successful project. Since the essence 
of Partnering is a combination of trust and open communication, the levels of trust and 
communication were measured from both the contractor and owner perspectives. The 
quality of the relationship can also be measured in the ease of the many repetitive 
delivery order negotiations and in the resolution of changes, claims, and disputes as they 


arise throughout the construction process. d 
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Finally, the overall satisfaction of the parties with the contract is the fourth key 
element and is the culmination of the parties’ effort in the first three elements. In the end, 
it is the most important, not only because it measures the combined success of 
construction, support, and relationships, but mainly because it determines the actions of 
the parties after the project is complete. A satisfied owner will likely expand the concept 
further, leading to better service and streamlined administration, while rewarding the 
good performance of the contractor with solid references or even award of more 
construction. The overall satisfaction of the parties is measured in three areas: 
satisfaction of the owner and contractor, satisfaction of the end-users of the 
construction, and the owner’s perception of efficiency as compared with other contract 


vehicles. 


Selection of Projects for Survey 


The search for potential sites was undertaken to collect as many data points as 
possible for the analysis. Established firms that specialize in JOC were contacted for 
potential sites. The Center for Job Order Contracting Excellence at Arizona State 
University also provided some potential sites for analysis. The Army and Navy provided 
points of contact. In addition, as sites were contacted to participate in the survey, new 
leads were generated. In the end, 35 sites met criteria as candidates for study. The sites 


are broken out as follows: 


U.S. Army/Corps of Engineers 14 sites 
State Universities 7 sites 
U.S. Air Force 6 sites 
U.S. Navy 5 sites 


Other Federal Government 3 sites 
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Two questionnaires were developed to measure contract performance at each 


Survey Development 


site. One was designed for the owner’s JOC project manager and the other was designed 
for the contractor’s project manager at the local site level. The evaluation consisted of 
two basic scoring systems, a numeric score (from | to 5, where 1 = Poor, 5 = 
Outstanding) and a multiple choice descriptive answer. The Owner and Contractor 


questionnaires are included as Appendices A and B, respectively. 


Several questions are duplicated in both questionnaires, as they measure areas 
common to both parties such as levels of trust and communication. Other questions were 
designed for the respondent to grade the performance of another party, such as the 
owner’s scoring of the contractor’s management effectiveness. In addition, respondents 


are asked to make comments about the JOC process at the end of the questionnaire. 


The owner’s questionnaire consists of twelve numeric scores and nine descriptive ( 
answers for a total of 21 questions. The contractor’s version contained three numeric ; 
scores and six descriptive answers for nine questions. Questions 13 and 14 of the 

owner’s questionnaire were not assessed for this study, as they were included to gather 


data for a separate research paper. 


The JOC/SABER Contractor Performance Questionnaire developed by Arizona 
State University’s Center for Job Order Contracting Excellence was used as a model for 


the development of these questionnaires. 


Data Collection 


70 questionnaires were mailed or faxed to respondents at the 35 sites from April 
14, 2000 to May 1, 2000. 63 questionnaires were returned by the cutoff date of May 19, 
2000, for a 90% response rate. After minor clarification on some survey data, the 


questionnaires were then segregated into the four population groups shown in Table 2. 


, 
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Table 2: Sample Population Summary 


17 69% 10% 





Non-Partnered 10 (28%) 7 (20%) | 
(en ee eee ee eee 


Of the 35 sites surveyed, both owner and contractor questionnaires were received 


for 28 sites, while at least one questionnaire was received for the remaining seven sites. 
As only one site was classified as a Partnered/Low bid site, there was insufficient data to 


analyze for that population and it was excluded from study. 


Chapter III: Analysis of Data 


The data collected from the survey responses was divided into three sample 
populations for analysis. Note that sample sizes for the two Non-Partnered groups are 
somewhat small and will be a concern during the statistical analysis. A summary of site 


totals and survey responses is depicted in Table 3. 


Table 3: Survey Response Summary 


| Sample Population | Number Owner Contractor Overall 


of Sites {| Questionnaires | Questionnaires | Response 


Received Received ; Rate 


Non-Partnered + 


RFP Source Selection 


Non-Partnered + 


Low Bid 





Survey response data was tabulated by sample population and by question. A 
summary chart displays the responses based on the two answer formats, numeric and 
descriptive. For numeric scores, the means for the three sample populations are 
presented side by side. For those questions where both contractor and owner supplied 
answers, both sets of data are presented on one chart. For descriptive answers, the 
number of responses for each answer is presented for all three sample populations 
studied. Due to the complexity of the chart type, only contractor or owner answers will ¢ | 


be shown on each chart. 
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The following chart abbreviations will denote sample populations: 


P-RFP: Partnered contracts procured via Request For Proposal (RFP) negotiated 


source selection 
NP-RFP: Non-Partnered contracts procured via RFP negotiated source selection 
NP-LB: Non-Partnered contracts procured through competitive bidding (low-bid) 
The following chart abbreviations will denote descriptive answers: 
VH: Very high MB: Much better than VR: Very reasonable 
Higher thannormal SB: Slightly betterthan’ R: Reasonable 


Average S: Same as SU: Somewhat 


unreasonable 
Le: Lower than normal SW: Slightly worse than  U: Unreasonable 


VL: Very Low MW: Much worse than EXC: Exceptional 


3.1 Performance of JOC Construction 


The performance of construction is analyzed to determine if a difference in 
performance is apparent by varying the procurement and administration methods. 
Owners were asked to grade the contractors’ efforts in quality, safety, on-time 
completion, subcontractor scheduling/performance, and warranty service. All answers in 
this element are numeric scores. Ideally, the combination of Partnering and RFP Source 


Selection should result in higher scores across the board in this element. 
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Quality of Construction 
According to the responses shown in Figure 1, it would appear that the RFP 
process does not affect quality but that Partnering has a tremendous effect on 
construction quality. This is somewhat surprising, since the goal of the RFP process is to 
ensure a higher quality product than the low bid process provides. This could indicate 
that the RFP process is ineffective or that Partnering has a much larger effect on quality 


than the procurement process. 


Quality of Construction 
Summary of Sample Means 


Sample Means 





P-RFP NP-RFP NP-LB 
Sample Population 


Figure 1: Quality of Construction Summary 


Comments from respondents generally did not address quality. Among 
Partnered/RFP projects, the few comments made were positive. One Non-Partnered/RFP 
owner specifically addressed a lack of sufficient quality control that led to rejection and 


rework of installed material. One Non-Partnered/Low bid contractor was praised for 


outstanding performance. 
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Contractor’s Safety Performance 


Data on contractor safety in Figure 2 actually shows that safety performance is 
slightly higher under low bid than RFP given a Non-Partnered environment. However, 
the Partnered/RFP environment seems to produce the safest job sites. This may be 
attributed to the increased level of communication on Partnered contracts as well as a 


common goal of an accident-free jobsite that many Partnering charters contain. 


Safety 
Summary of Sample Means 





P-RFP NP-RFP NP-LB 
Sample Population 


Figure 2: Safety Summary 


On-Time Completion of Job Orders 


The distribution of responses in Figure 3 fits the expected pattern in this area. 
RFP contracts perform slightly better than low bid contracts in a Non-Partnered 
environment, but the addition of Partnering results in a much higher performance score. 
This may be attributed to the tendency of Partnering to resolve issues disputes quickly at 
the lowest levels before a dispute can cause costly delays to the work in question. The 


increased communication at the beginning of the project may also prevent modifications 
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that delay the project. 
On-Time Completion 
Summary of Sample Means 
3.8 a8 











P-RFP NP-RFP NP-LB 
Sample Population 


Figure 3: On-Time Completion Summary \ 
Comments were not directed at completion time, but a couple of comments from 


Non-Partnered/Low Bid sites indicated that the process is slowed if the government has 


tight restrictions over design. 
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Subcontractor Scheduling and Performance 


The results in Figure 4 indicate a much higher performance level in a 
Partnered/RFP environment than in the other two formats. Traditionally, JOC 
contractors staff only the construction management personnel in-house, subcontracting 
for trades in the execution of delivery orders. This is mainly due to the uncertainty of the 
type of work issued under a JOC. Usually, the JOC contractors will try to develop 
relationships with the better subcontractors in the area, as better work can be expected 
with less hassle and risk than using less qualified subs. In the JOC business, better work 
leads to more work. If a low coefficient does not allow use of better contractors, 


problems may result. 


Scheduling/Performance of Subs 
Summary of Sample Means 


Sampie Means 





P-RFP | NP-RFP ~ NP-LB 
Sample Population 


Figure 4: Scheduling/Performance of Subcontractors Summary 


One contractor in a Partnered/RFP environment cites “working with 
subcontractors that we haven’t been comfortable with” as a reason for a slow start to a 
contract. A Non-Partnered/RFP owner blamed “a lot of unqualified subcontractors” for 
significant rework. A Non-Partnered/Low Bid owner drew a direct conclusion between 


an artificially low coefficient, unproven subcontractors, and user dissatisfaction. 
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Warranty Service 
Warranty Service 
Summary of Sample Means 
4.5 
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P-RFP NP-RFP NP-LB 
Sample Population 


Figure 5: Warranty Service Summary 
A 


The trend in Figure 5 is expected, as source selection seems to add improvement “ 
over low bid, then Partnering results in even higher performance. Basically, the key here 
is money. Often, a low coefficient allows for enough money to complete the work, but 
little or no contingency for rework. Since some warranty issues are a grey area at best, it 
can become quite contentious when nobody owns up to responsibility for fixing an item 


under warranty. Quite possibly, the Partnering environment may lead to quicker 


resolution of warranty issues and therefore higher performance. 


3.2 Performance of Construction Support 


The contractual support that the parties in a construction project provide each 
other with is critical in achieving a successful outcome. Decision-making forces rely on 


the information and support provided to keep the project on schedule and within budget. 
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The true value of the management of a JOC on both sides of the contract rests with the 
ability to get ahead of the project and stay ahead of it though its duration. The following 


areas will measure the performance in this critical area. 


Responsiveness and Timeliness of Administrative Support 


This area was intended to measure the responsiveness of the owner and the 
contractor in providing support to each other. The owner was asked to grade the 
contractor’s responsiveness and timeliness with cost estimates and drawings, while the 
contractor was asked to grade the owner’s responsiveness with support such as requests- 


for-information (RFI) and submittals. 


Responsiveness of Support 
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~P-RFP NP-R NP-LB 
Sample Population 


Figure 6: Responsiveness of Support Summary 


The owner responses in Figure 6 fit the general expected pattern. However, the 
contractor answers vary from the expected trend. In theory, the trends of the owner and 
the contractor should be somewhat similar. In rereading both survey questions, I realized 
that the owner’s questionnaire was straightforward in nature, while J noticed that the 


contractor question could be inferring that the contractors should grade themselves on 


as ( 


the support they provide to the owner in terms of RFIs and submittals. I suspect that the 


vague wording of the question could have led to the irregular data pattern. 


Innovation and Value Engineering 


Innovation and Value Engineering 
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Figure 7: Innovation and Value Engineering Summary 


The owner was asked to grade the contractor on the quality and frequency of 
innovative ideas and Value Engineering proposals, while the contractor was asked to 
grade the owner on the reception to innovation and Value Engineering proposals. The 
intent was to measure the climate for innovation among the various formats. There ts a 
consistent connection between the contractor and owner answers here, as expected, so 
the data looks reasonable. Based on the data in Figure 7, there appears to be a healthy 
climate for innovation in the Partnered format that the other formats do not seem to 


share. The Partnered owner and contractor scores are also much closer together, 


indicating more of a consensus view. 
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Responsiveness to Client Needs 


This question is measuring the ability of the contractor to respond to the needs of 
the clients using the JOC. The Partnered contracts in Figure 8 show much higher scores 
than the other two formats, which may be attributed to the greater involvement of the 
end-user in determining the scope of the work and greater communication among the 


project stakeholders. 


Responsiveness to Client Needs 
Summary of Sample Means 
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Figure 8: Responsiveness to Client Needs Summary 


As for the drop in the Non-Partnered/RFP format score, this can be attributed to 
two sites that had problems with this very area. The first owner commented that the 
contractor came in with a limited staff into an environment where they were in direct 
competition with in-house forces. The feeling was that they should be doing more to 
please the clients in order to garner more work. The second owner was an Air Force 
owner working through an Army agency who sees “a large communications gap from 
the Air Force’s request to the contractor’s delivery.” This could be due to the structure 


of the contract organization vice the procurement method of the contract. 


wo ¢ 
Ability to Prevent and Solve Scheduling and Site Coordination Problems 


The owner was asked to evaluate the ability of the contractor to prevent and 
solve scheduling and site coordination problems. This is a key area of concern for a JOC 
since many projects involve repair or renovation of an existing facility and are of an 
urgent nature where construction work takes place in shorter time frames. This summary 
(Figure 9) shows that there is a much higher score for Partnered projects and very little 


difference between the Non-Partnered formats. 
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Figure 9: Preventing and Solving Problems Summary 


Though not fitting the expected pattern for Non-Partnered/RFP, the Partnered 
scores were expected much higher as this is one of the main strengths of Partnering. 
Many problems with construction site coordination and scheduling can be traced to 
miscommunication or non-communication. Also, a lack of mutual trust leads the parties 
to suspect one another when a problem creeps up and may inhibit communication. With 
Partnering, scheduling and coordination problems are more likely to be aired quickly and 
resolved due to the open nature of the relationship. An owner who specifically identified 
a communication problem in a JOC contract also stated “It seems that scheduling é 


problems, site coordination problems, and dissatisfaction is the norm.” 
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Contractor’s Management Effectiveness 


The management effectiveness was evaluated due to its significance in JOC. The 
contract can be difficult to manage since it is made up of many small delivery orders 
comprising many different types of work. Often, there are dozens of jobs going on 
simultaneously in widespread locations throughout a site. On top of that, the urgency of 
many JOC jobs requires expedited materials, flexible scheduling requirements, and 
working around tenants. The owner’s perception of the contractors management 
effectiveness is a key indicator of a successful project. The Partnered sample population 


in Figure 10 has much higher test scores in this field, with the Non-Partnered scores 


essentially the same. 
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Figure 10: Management Effectiveness Summary 


Level of Administrative Effort 
In theory, the owner’s administrative effort should be reduced in a Partnered 
environment, since the need for dispute-oriented documentation is greatly eliminated. 


Though there is an additional effort required for Partnering, it is seen as an up-front 


o ( 


investment that should save time later in the project by preventing problems and quickly 
resolving those that do come up. Also, RFP projects should have reduced administration 
costs due to the selection of a qualified contractor and presumably an adequate 


coefficient. 


Analysis of the responses, shown in Figure 11, indicates that over half of the 
owners on negotiated contracts assessed the administrative effort as average or better. 
Over half of the owners on low bid contracts rated the administrative effort as higher 


than normal. 


Owner Administrative Effort 
Summary of Responses 
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Figure 11: Owner Administrative Effort Summary 


In Figure 12, the contractors’ responses on their level of administrative effort are 
summarized. In all three types of contracts, over half the contractors rated the amount of 
administrative effort as higher than normal. This indicates that the contractors perceive 


very little difference in their administrative workload based on the contract format. 


43 


Contractor Administrative Effort 
Summary of Responses 





Number of Responses 





Answer Category 


Figure 12: Contractor Administrative Effort Summary 


Reasonableness of Owner’s Inspection and Quality Assurance Programs 


Reasonableness of Inspection 
Summary of Responses 





Number of Responses 





Answer Category 


Figure 13: Reasonableness of Inspection Summary 


Figure 13 shows a summary of responses from contractors who were asked to 
grade the reasonableness of the owner’s inspection and Quality Assurance programs. 


The contract format theoretically has little to do with the reasonableness of the owner’s 


a 
inspection and Quality Assurance efforts. After all, in JOC contracts with government 
employees conducting oversight, the main focus is on fair and accurate assessments of 
the contractors quality control program. It would be expected that in the majority of 
cases, the contractors would, at a minimum, view the owner’s efforts as usually fair and 
accurate, with an occasional inaccurate assessment. This definition fits the reasonable 


category in Figure 13 and the highest-frequency answers were at least reasonable, with 


Non-Partnered/Low Bid having a majority in the very reasonable category. 


However, in situations where trust is low, owner representatives tend to increase 
inspection efforts in an attempt to “keep the contractor honest.” Though sometimes the 
increase in monitoring is warranted, in other cases overzealous inspection may result, 
with a corresponding decrease in the quality of the contractor-owner relationship. It is 
interesting to note here that, although the Partnered responses outnumber the Non- 

Partnered responses 16 to 15, the number of Non-Partnered respondents who stated that 
\ 


the owner’s efforts were unreasonable or somewhat unreasonable outnumbered their 


Partnered counterparts by a ratio of five to one. 


3.3. Relationships 


The maintenance of a good relationship between the owner and contractor is 
essential to the success of a JOC. The repetitive nature of job order negotiations, 
combined with the fairly open structure of the contract, means that parties that get along 
well in a team environment will do much better in a JOC than those who thrive on the 
well defined structure of the traditional competitively bid contract. For some, it is a 
welcome change from the adversarial roles inherent in Jump sum contracting. For others 
who are resistant to change, the attempts to administer JOC projects with a competitive 


bid mindset tend to result in hard feelings that make collaboration and negotiation 


difficult. 


High levels of trust and communication, mutually beneficial negotiation, and @ 
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quick resolution of disputes are all indicators of a successful JOC contract, or any 
construction contract for that matter. Since Partnering was designed to strengthen 
performance in these areas, it is expected that higher scores will be particularly evident in 
the Partnered/RFP sample population. As for Non-Partnered/RFP contracts, it is 

expected that dispute resolution and ease of negotiations would be enhanced since the 


unit prices would allow the contractor a reasonable profit. 


Ease of Individual Job Order Negotiations 


The trends in Figure 14 are in line with expectations. The most difficult 
negotiations from both owner and contractor perspectives are in the Non-Partnered/Low 
Bid arena. Scores increase as low bid 1s replaced by RFP, then further increase with the 


addition of Partnering. Three interesting items are apparent from this summary. 
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Figure 14: Ease of Job Order Negotiations Summary 
First, the contractors across the board perceive negotiations are more difficult in 


JOC than the owners. This could be due to the fact that the contractors expend much 


more effort in a given negotiation than the owners do, as they prepare the proposal and 
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must justify the requirements for line items to the owner. This might also indicate that 
owners find JOC negotiations easier in general compared with their experience in more 
adversarial forms of contracting, while the contractors are basing their expectations on 


their similar experience with other JOC projects. - 


Second, the contractor score trend from Non-Partnered/Low Bid to 
Partnered/RFP is approximately linear, suggesting that the contractors’ perceptions of 


negotiation are improved equally by the additions of source selection and Partnering. 


Third, the owner scores increase the most when RFP source selection replaces 
low bid procurement, then slightly increase with the addition of Partnering. This would 
indicate that the most significant factor to the owner’s perception of successful job order 


negotiations is the method of contractor selection. 


Resolution of Changes, Claims, and Disputes ( 


Both the owners and contractors were asked the same question on the resolution 
of changes, claims, and disputes that arose within their contract. The descriptive answers 
were amplified with definitions. An answer of exceptional meant that issues were quickly 
resolved with mutual benefit. Good meant most were quickly resolved with mutual 
benefit, though a few required lengthy negotiations. Fair was similar to good, except a 
few issues required more formal dispute resolution techniques, such as dispute review 
boards and mediation. Finally, an answer of poor indicated that the parties consistently 
required lengthy negotiation, formal dispute resolution, or litigation to resolve issues. 
Figures 15 and 16 reflect the owner and contractor perceptions of dispute resolution, 


respectively. 
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Dispute Resolution-Owner View 
Summary of Sample Means 
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Figure 15: Dispute Resolution Summary - Owner View 


The inherent nature of the JOC process encourages early dispute resolution, since 
the owner retains the option of issuing work under a JOC or through another contracting 
vehicle. If a JOC is plagued with unresolved disputes or pending claims, the owner is less 
likely to issue future delivery orders. Most large JOC contractors maintain a policy of 
using claims or litigation as a last resort in recognition of this fact, and unless significant 
sums of money are at stake, it is generally recognized that a loss on one or two delivery 
orders is preferable to 30 to 40 lost delivery orders. Therefore, the expectation was to 


see very few answers below fair, with most in the good category. 


Dispute Resolution-Contractor View 
Summary of Responses 
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Figure 16: Dispute Resolution Summary - Contractor View 


as 


The summary of responses in Figures 15 and 16 are in line with expectations. 


Over half of the respondents rated dispute resolution as good or better. Half the owners 


using Partnered/RFP contracts rated dispute resolution as excellent. The single poor 


response came from a Non-Partnered/RFP contractor. 


Level of Trust 


Both owners and contractors were posed the same question on the level of trust 


that exists among project participants as compared with experience in similar contracts, 


or their expectations if this was the first JOC contract at the site. The five descriptive 


answers ranged from very low to very high. 


An analysis of the summaries of responses for both owner and contractor in 


Figures 17 and 18 both indicate that the Partnered group has higher levels of trust than 


both the Non-Partnered groups. Within the Non-Partnered groups, owners perceived ¢ 
, 


higher levels of trust in the RFP contracts as compared with the low bid contracts. 
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Figure 17: Level of Trust - Owner Perspective Summary 





From the owner’s perspective, most responses for both RFP groups were higher 


than normal or very high, while over half in the Non-Partnered/Low Bid group rated é 
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trust as average or low. 81% of Partnered/RFP owners felt their sites maintained a 
higher than normal or very high level of trust, compared with 63% of Non- 
Partnered/RFP sites and 33% of Non-Partnered/Low Bid sites. 


Level of Trust 
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Figure 18: Level of Trust - Contractor Perspective Summary 


From the contractor’s point of view, the highest frequency answers were the 
same as the seen in the owners’ responses. 75% of Partnered/RFP contractors perceived 
a higher than normal or very high level of trust, compared with 44% of Non- 
Partnered/RFP sites and 33% of Non-Partnered/Low Bid sites. 


Comments from many respondents emphasized the importance of trust in the 
success of a JOC contract. In particular, contractors from Partnered/RFP sites were the 


most vocal in its promotion and made the following comments: 


- “Continuous building of ‘TRUST between contractor/owner/customer is crucial. 


Give & take is necessary to make the contract work.” 


- “The University is an honest and straightforward owner. It is these two 
qualities that make this a workable relationship. If they were not open and trustworthy, 


we would not be able to respond to their need as promptly, effectively, and efficiently.” 


so 


- “High levels of trust and great partnership help to accomplish our common 


goals.” 


- “The reason that the JOC has been so successful here is because of the trust 


that has been developed between the contractor, Corps of Engineers, and the owner.” 


Trust is valued from Non-Partnered sites as well. One Non-Partnered/RFP 
contractor replied, “Although there 1s no formal Partnering process utilized on this 
project, there is a strong “Team’ working relationship with our government counterparts. 


Building and maintaining trust is the key to the success of a JOC.” 


Level of Communication 


As with level of trust, the level of communication was measured from both the 
contractor and owner perspectives. Again, answers ranged from very low to very high. 
Figures 19 and 20 summarize the responses, which both demonstrate that the Partnered q 
group has higher levels of communication than both the Non-Partnered groups. Within 
the Non-Partnered groups, owners perceived higher levels of communication in the RFP 


contracts when evaluated against the low bid contracts. An impressive 94% of 
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Figure 19: Level of Communication - Owner’s Perspective Summary y 
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Partnered/RFP owners felt their sites maintained a higher than normal or very high level 
of communication, compared with 50% of Non-Partnered/RFP sites and 33% of Non- 
Partnered/Low Bid sites. 75% of Partnered/RFP contractors perceived a higher than 
normal or very high level of communication, compared with 56% of Non-Partnered/RFP 
sites and 33% of Non-Partnered/Low Bid sites. 
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Figure 20: Level of Communication - Contractor Perspective Summary 


This comment from a Non-Partnered/RFP owner underscores the importance of 
communication: “JOC requires constant communication and close monitoring since we 
lack a formal set of drawings and specifications. The technical specs that are 
incorporated into the contract are so generic they have little impact. Questions regarding 
materials, methods, scheduling, field adjustments, and customer-generated requests for 


changes must constantly be addressed.” 
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3.4 Measurement of Overall Satisfaction 


Though much data has been studied on individual performance factors of the 
JOC process, the most significant measurement of contract success is taken from the 
standpoint of overall satisfaction with the project. A symphony may have many talented 
musicians who play flawlessly, but unless all the instruments are playing well and the 


conductor orchestrates them properly, the audience simply will not enjoy it. 


In the case of a construction contract, the audience per se is the end-user of the 
JOC’s construction services and the one that will live with the results long after the 
contractor demobilizes. In addition to measuring the end-users satisfaction, the 
contractor and owner also must provide their opinion on how successful the contract has 
been. Theoretically, Partnering and source selection should enhance the satisfaction 
levels of both the contractor, since disputes will be reduced in number and resolved ( 


quicker, hopefully leading to a mutually beneficial result. 


The efficiency of JOC will be compared with other methods to see if Partnering 


and source selection enhance the efficiency of the JOC process. 


Customer (End-user) Satisfaction 


The sample means displayed in Figure 21 show that the Non-Partnered/RFP 
scores are superior to Non-Partnered/Low Bid, with the Partnered/RFP mean much 
higher than the two other populations. The trend is approximately linear. The clear 
implication is that the Partnered/RFP format results in the highest levels of customer 


satisfaction, though in a Non-Partnered environment, RFP is superior to Low Bid. 
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Customer (End-user) Satisfaction 
Summary of Sample Means 
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Figure 21: Customer Satisfaction Summary 


Owner and Contractor Overall Satisfaction 


The owner responses presented in Figure 22 clearly indicate much higher levels 
of satisfaction with Partnered/RFP contracts. More significantly, however, 100% of 
Partnered/RFP owners surveyed report Higher-than-normal or Very-high levels of 
satisfaction, compared with 63% of Non-Partnered/RFP owners and 33% of Non- 


Partnered/Low Bid owners. 
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Figure 22: Overall Satisfaction Summary - Owner 
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Figure 23: Overall Satisfaction Summary - Contractor 


As expected, the contractor responses presented in Figure 23 parallel the owner 
answers. 94% of the contractors at Partnered/RFP sites reported higher than normal or 
very high levels of satisfaction, compared with 44% and 33% in the Non-Partnered/RFP 
and Non-Partnered/Low Bid populations, respectively. Once again, the Partnering/RFP 


sites display much higher levels of overall satisfaction with the contract. 


Efficiency of JOC contracts 
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Figure 24: JOC Efficiency Summary 
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Since the JOC process is inherently more efficient for the completion of small 
projects than traditional sealed bid methods and other types of contracting, the data in 
Figure 24 was expected to slant toward the “better than” side of the scale. 69% of 
Partnered/RFP sites were rated much better than other contracting methods as compared 


with 50% for Non-Partnered/RFP and 33% for Non-Partnered/Low Bid. 


Comments from Respondents . 


Though comments made at the end of the questionnaire were not requested for 
specific comparison, the respondents expressed some valuable information that 
illuminated the response received. Where possible, comments that were germane to an 


area under analysis were presented along with the data. 


However, one area of interest discussed more than once in the comments that 
was not part of the study concerned the number and stability of the personnel involved in 
the JOC process. Respondents specifically identified this area as having a major impact 
on the success of the project. The following comments were made on the subject of 


staffing and administration: 


- Contractor: “We have been here for 8 years working with the same people day 


in and day out and we have become a team.” 


- Contractor: “We have a very good working relationship with the Army and 
COE. I attribute our continued success (2 JOC projects) to stability of the [contractor] 


team, our knowledge of construction, and our close relationships with the owner.” 


- Contractor: “The process of construction demands consistency in expectation 
by the contractor from the owner (gov). Probably the greatest deterrent to a successful 
Partnering is how each SABER contracting officer interprets their duty requirements of 
the contractor. With each new change of personnel, a new way of doing things is 


implemented, some better than others but a moving target at best.” 
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- Contractor: “We have had to work hard at this contract, probably because of 
the number of people involved. We are working with 4 Corps of Engineer districts in 5 
different states. Negotiations are held with quite a large number of participants. 
Inspections, as you can imagine, can be trying.” 

- Owner: “The JOC process is a good one if the contractor takes a serious 


interest in its success. We have been exposed to about four project managers during the 


9 months of this contract and at times we were unsure of who was truly in charge.” 


- Contractor: “[OQwner] has the most frequent turnover of both contracting & 


CE staff and the vast majority are trainees.” 


Clearly, both owner and contractor should attempt to keep turnover to a 
minimum in JOC and aim for consistency in contractual expectations by keeping the 


number of personnel down to a manageable level. 


3.5 Statistical Analysis 


The data collected through the survey responses was subjected to analysis to 
determine its statistical significance. A numerical score was first assigned to every 
answer. For those questions that had already been reported as numbers, they were scored 
as reported. For descriptive answers, numbers were assigned to each category according 


to the following convention: 

- For answers ranging from “Very Low” to “Very High”: 1-5 
- For answers ranging from “Poor” to Exceptional: 1-4 
- For answers ranging from “Much Worse Than” to Much Better Than”: 1-5 


Next, a series of histograms was developed to determine if a normal distribution 
of data existed for answers given by a certain sample population. If two sets of data to be 


compared were of a normal distribution, a comparison of small sample means via the t- 
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test was used. If the data did not meet that criteria, the Mann-Whitney U-Test of sample 


medians was employed. 


In comparing the data, the response scores for each question was analyzed in 


three separate tests: 
1. Non-Partnered/RFP vs. Non-Partnered/Low Bid 
2. Non-Partnered/RFP vs. Partnered/RFP 
3. Partnered/RFP vs. Non-Partnered/Low Bid 


This resulted in 84 comparisons (28 questions x 3 tests). Of all the comparisons 
conducted, only three could be performed by the small sample means test, with the rest 
conducted by the Mann-Whitney U-Test. It was decided to run all 84 comparisons 
through the Mann-Whitney U-Test as well as perform the three small sample means 


tests. All tests were conducted with a significance level of 0.05. 


The development of a decision rule centered around a one-tail test. Three 


different decision rules were developed according to the expectations of the responses: 


Decision Rule #1 - Non-Partnered/RFP vs. Non-Partnered/Low Bid 


Null Hypothesis: Performance of Non-Partnered/RFP projects is the same as or inferior 
to Non-Partnered/Low Bid projects. 


Alternate Hypothesis: Performance of Non-Partnered/RFP projects is superior to Non- 
Partnered/Low Bid projects. 


Decision Rule #2 - Non-Partnered/RFP vs. Partnered/RFP 


Null Hypothesis: Performance of Partnered/RFP projects is the same as or inferior to 
Non-Partnered/RFP projects. 


Alternate Hypothesis: Performance of Partnered/RFP projects is superior to Non- 
Partnered/RFP projects. 
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Decision Rule #3 - Partnered/RFP vs. Non-Partnered/Low Bid 


Null Hypothesis: Performance of Partnered/RFP projects is the same as or inferior to 
Non-Partnered/Low Bid projects. 


Alternate Hypothesis: Performance of Partnered/RFP projects is superior to Non- 
Partnered/Low Bid projects. 


The three small sample means tests did not produce statistically significant 
results. Of the 84 Mann-Whitney U-Test comparisons conducted, there were only 13 
cases where the data was statistically significant and the null hypothesis was rejected. 


These cases are presented in the form of the alternate hypothesis: 


Non-Partnered/RFP vs. Partnered/RFP 


1, Owner Question #3: The quality and frequency of innovative ideas and Value ‘ 
Engineering proposals from contractors within Partnered/RFP contracts are superior to 


Non-Partnered/RFP contracts. 


2. Owner Question # 4: Contractor responsiveness to client needs within Partnered/RFP 


contracts is superior to Non-Partnered/RFP contracts. 


3. Contractor Question # 7: The level of trust within Partnered/RFP contracts is superior 


to Non-Partnered/RFP contracts. 


4. Contractor Question # 9: The overall satisfaction of contractors within Partnered/RFP 


contracts is superior to Non-Partnered/RFP contracts. 
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Partnered/RFP vs. Non-Partnered/Low Bid 


5. Owner Question #1: The responsiveness and timeliness of cost estimates and “basic” 
drawings from contractors within Partnered/RFP contracts 1s superior to Non- 
Partnered/Low Bid contracts. 


6. Owner Question #8: The contractors’ ability to prevent and solve scheduling and site 
coordination problems within Partnered/RFP contracts is superior to Non- 


Partnered/Low Bid contracts. 


7. Owner Question #10: Customer (end-user) satisfaction within Partnered/RFP 


contracts is superior to Non-Partnered/Low Bid contracts. 


8. Owner Question #11: The management effectiveness of contractors within 


Partnered/RFP contracts 1s superior to Non-Partnered/Low Bid contracts. 


9. Owner Question #17: The required level of administrative effort within Partnered/RFP 
contracts 1s less than within Non-Partnered/Low Bid contracts. 


10. Owner Question #18: The level of trust within Partnered/RFP contracts is superior 
to Non-Partnered/Low Bid contracts. 


1]. Owner Question #19: The level of communication within Partnered/RFP contracts is 


superior to Non-Partnered/Low Bid contracts. 
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12. Owner Question #20: The owners’ level of overall satisfaction within Partnered/RFP 


contracts 1s superior to Non-Partnered/Low Bid contracts. 


13. Contractor Question #9: The contractors’ level of overall satisfaction within 


Partnered/RFP contracts is superior to Non-Partnered/Low Bid contracts. 
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Chapter IV: Conclusions and Recommendations 


Conclusions 


Job Order Contracting is a very popular method with facility owners who need a 
responsive method for accomplishing small construction and repair projects in a 
streamlined manner. However, the relationships of the parties in such an open and 
flexible structure plays a much greater role in the project’s success as compared to more 
traditional contracting methods. The formal Partnering process that serves to strengthen 
and focus these relationships toward a common vision of success should therefore have a 
significant impact on the performance of JOC contracts. In addition, the selection of a 
qualified, responsive JOC contractor through a source selection process should yield 


better performance then can be gained through competitive bid. 


Though this study did not produce a large volume of statistically significant data 
with which to draw sweeping inferences, a few key areas studied did show that owners 
and contractors feel that Partnering and RFP source selection add value to the JOC 
process. Among projects that were procured through RFP source selection, Partnered 
contracts had significantly higher levels of contractor innovation, contractor 
responsiveness to client needs, trust among project participants, and contractor 


satisfaction. 


When Partnered contracts procured through RFP source selection were 
compared with Non-Partnered, low competitive bid contracts, the results were more 
impressive. From the owner’s perspective, the contractors in the Partnered/RFP format 
were much more responsive with cost estimates and required drawings, displayed greater 
management effectiveness, and were more able to solve and prevent scheduling and site 
coordination problems. In addition, Partnered/RFP owners reported a reduced level of 
administrative efforts in their projects. Among project participants, those in the 
Partnered/RFP population enjoyed higher levels of trust and communication than their 
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Non-Partnered/Low Bid peers. 


The most significant result in the comparison of Partnered/RFP projects to Non- 
Partnered/Low Bid projects was in the area of overall satisfaction. A consensus of the 
three different groups involved in the process, owner, contractor, and end-user, all 
reported much higher levels of satisfaction under the Partnered/RFP format. Since this 
takes into account how all the various elements of the contract perform as a whole, it is 


very strong endorsement of Partnering and source selection. 


The results can be summed up in an especially poignant comment from an owner 


in the Partnered/RFP group: 


- “J have been involved with JOC going on 11 years. The lesson that I have 
learned is to have a good relationship with the contractor and have a contractor with a 
good track record....Do some calling, check other places where the contractor has 


worked, and ask about their performance. This could save a lot of problems later on.” 


Recommendations 


Based on the study results, facility managers who want to have a responsive and 
cost-effective contract that is relatively easy to administer and generates high levels of 
customer satisfaction need to select JOC contractors through a process of source 
selection that values past performance and qualifications. Artificially low coefficients 
should be avoided. Once a contractor is selected, the owner and contractor should 
implement a robust Partnering program that is continually championed and reinforced. 
Personnel changes should be kept to a minimum, but new personnel should be 


indoctrinated as to the common goals in the Partnering charter as they join the team. 


Many of the Mann-Whitney U-Tests that were run and did not produce 
significant statistical data were very close. In fact, 24 of the 71 failed tests (34%) scored 
90% or more of the points required to prove statistical significance. This is attributed to 


the small sample size in the Non-Partnered populations. A further study could expand the 
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number of sites included in the survey and possibly produce more statistically significant 
results. 
The primary basis for this study was subjective opinions of contractor 


performance. Another study might include objective measures of performance such as 


cost escalation, on-time completion of delivery orders, and other similar metrics. 
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Appendix A: Owner JOC/SABER Questionnaire 


————— ae 





Date conta! Avance Length: Base years: Option Years: 
Current year: Maximum annual contract value:$ 
| Number of JOC/SABER contracts on site:____——s-§ Number fi contractor management staff: 
| Type of estimating manual (check one): Q Army UPB OR.S.Means OQ Other: 


Is a formal Partnering process utilized on this contract? O) Yes O No 


| Contract Procurement Method: 


OC) Lowest bidder: Full and open competition without pre-qualification 
Q Lowest pre-qualified bidder: Lowest bid selected after pre-qualification process (Select bidder’s list) 


QO) Negotiated: Source selection process with weighted evaluation based on price and other factors 


CQ) Other: 


Contractor Performance Ratings (1 = Poor, 5 = Outstanding) Rating 
1. Responsiveness and timeliness of cost estimates and “basic” drawings: (1-5) 
2. Ease of individual job order negotiations: (1-5) 


3. Quality/frequency of innovative ideas and Value Engineering proposals: (1-5) 


4. Responsiveness to client needs: (1-5) 
5. Quality of construction: (i-5) 
6. Contractor’s safety performance: (1-5) 
7. Subcontractor scheduling and performance: c=5)" 
8. Ability to prevent and solve scheduling and site coordination problems: (1-5) 
9. On time completion of job orders: (1-5) 
10. Customer (end-user) satisfaction: (1-5) 
11. Contractor’s management effectiveness: (1-5) 
12, Warranty service: (1-5) 


Team Performance Questions (check most applicable box) 

13. Adequacy of contractor coefficient: 

Very High: High prices of proposals limit our ability to issue delivery orders under contract 

High: Prices for proposals seem slightly high as compared to local construction costs 

Reasonable: Prices for proposals reasonably in line with local construction costs 

Low: Prices for proposals seem slightly low as compared to local construction costs 

Very Low: Low coefficient seems to negatively impact contractor’s performance/profitability 
Page | of 3 
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14. Level of design required for average delivery order: 

Same level that is required for standard bid procurements (full plans/specs) 

Owner develops partial plans and specifications that are abbreviated for JOC/SABER 

Owner prepares limited design based on jointly developed scope of work 

Contractor prepares limited design based on jointly developed scope of work, owner approves 
No design, just a basic schematic and an jointly developed scope of work 


eS of 


5. Resolution of changes, claims, and disputes: 

Exceptional: Parties consistently resolve issues quickly to the benefit of all concerned 
Good: Most issues resolved quickly with mutual benefit, though a few require lengthy 
negotiation 


Fair: Most issues resolved either quickly or with lengthy negotiation, but a few require more 
formal dispute resolution techniques (mediation, review boards, etc...) 


Poor: Parties consistently require lengthy negotiation, dispute resolution, or litigation to resolve 
issues 


i» 
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16. Overall value of contractor’s construction services as compared to cost, considering 
responsiveness, quality, customer satisfaction, and other intangible elements: 


Exceptional: The services provided by the contractor add tremendous value for the cost 
Good: We receive sufficient value for our construction dollars 

Fair: JOC is good for some jobs, but we get better value through other procurement means 
Poor: JOC/SABER is too expensive for the value we receive 


Oooo 


17. Level of owner’s administrative effort as compared with experience in similar contracts (or 
expectations, if first JOC/SABER contract): 


Q Very high Q Higher than normal QO) Average Q Lower than normal QO Low 


18. Level of trust among project participants as compared with experience in similar contracts (or 
expectations, if first JOC/SABER contract): 


Q Very high Q) Higher than normal O) Average ©) Lower than normal QO) Low 


19. Level of communication among project participants as compared with experience in similar 
contracts (or expectations, if first JOC/SABER contract): 


Q Very high Q) Higher than normal OC) Average QO) Lower than normal QO) Low 


20. Overall level of satisfaction with this JOC/SABER contract: 
Q) Very high Q Higher than normal QO) Average OQ) Lower than normal QO Low 


21. How do you rate the efficiency of your JOC/SABER contract compared to other methods of 
project delivery? 
QO) Much better than OQ Slightly better than QO) Sameas OSlightly worse than QO Much worse than 
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Comments about the JOC process at your site: 
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Thank you very much for your time. If you would like a copy of the survey results, please check the box 


below and provide an address. Again, your support of this effort is greatly appreciated. 
() I would like a free copy of the survey results. Please send to: 
Name: 


Address: 
City, State, Zip: 


Control # JOC- 
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Appendix B: Contractor JOC/SABER Questionnaire 
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Date contract awarded: Length: Base years: Option a 
Current year: Maximum annual contract value:$ 

Number of JOC/SABER contracts on site:___ = Numberof contractor management staff: 

| Type of estimating manual (check one): Q Army UPB (2 R.S. Means Q) Other: 

Is a formal Partnering process utilized on this contract? QO) Yes OO No 

Contract Procurement Method: 

QO Lowest bidder: Full and open competition without pre-qualification 


OQ Lowest pre-qualified bidder: Lowest bid selected after pre-qualification process (select bidder’s list) 


OQ) Negotiated: Source selection process with weighted evaluation based on price and other factors 


C) Other: 


Owner Performance Ratings (1 = Poor, 5 = Outstanding) Rating 

1. Ease of individual job order negotiations: (1-5) 

2. Responsiveness and timeliness of owner support (RFI, submittals, etc...): (1-5) 

3. Reception to innovation and Value Engineering proposals: (-5) 

Team Performance Questions (check most applicable box) é 
4. Reasonableness of owner’s inspection/Quality Assurance efforts: 

OQ Very reasonable: Consistently fair and accurate assessment of quality control and workmanship 

OQ Reasonable: Usually fair and accurate, occasionally making unfair and inaccurate assessments 

QO) Somewhat unreasonable: Varies between fair/accurate and unfair/inaccurate assessments 

QO Unreasonable: Consistently unfair and inaccurate assessment of quality control and 


workmanship 


5. Resolution of changes, claims, and disputes: 
Q Exceptional: Parties consistently resolve issues quickly to the benefit of all concerned 


Q Good: Most issues resolved quickly with mutual benefit, though a few require lengthy 
negotiation 


Q Fair: Most issues resolved either quickly or with lengthy negotiation, but a few require more 
formal dispute resolution techniques (mediation, review boards, etc...) 


Q) Poor: Parties consistently require lengthy negotiation, dispute resolution, or litigation to resolve 
issues 


6. Level of Contractor’s administrative effort as compared with experience in similar contracts: 


Q) Very high Q) Higher than normal Q Average Q) Lower than normal Q) Low 
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7. Level of trust among project participants as compared with experience in similar contracts: 
Q) Very high O Higher than normal CL) Average Q) Lower than normal QO) Low 


8. Level of communication among project participants compared with experience in similar 
contracts: 


Q) Very high QO) Higher than normal QO) Average QO Lower than normal QO) Low 
9. Your overall satisfaction with contract: 
Q) Very high Q) Higher than normal QO) Average Q) Lower than normal QO) Low 


Comments about the JOC process at your site: 


Thank you very much for your time. If you would like a copy of the survey results, please check the box 
below and provide an address. Again, your support of this effort is greatly appreciated. 


Q) I would like a free copy of the survey results. Please send to: 
Name: 

Address: 

City, State, Zip: 


Control # JOC- . Page 2 of 2 
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Appendix C: Summary of Survey Comments 
Partnered and Best Value contracts 


Owner comments: 


“The JOC has turned into a very adaptable, catch-all type of tool. In today’s age 
of dwindling budgets and the necessity to obligate funds quickly, the JOC has become 
the choice for fast-tracking small projects. This has also caused the use of JOC to stretch 
outside its intended purpose. Because of time constraints, JOC has been used in place of 
SAPs and low-bid IFBs. Since there has been no real research that I have seen on the 
cost of JOC compared to other vehicles, it is hard to say if stretching the roles of JOC is 
good or bad. Either way, it seems to be a necessity to accomplish tasks in the current 
fiscal environment. The bottom line is that the customers are very satisfied with the JOC 


contractor and request them all the time.” 
“JOC has its place when the project 1s fast-tracked.” 


“The JOC contract is good and works well for the government. Some 


administrative problems exist.” 


“JOC contract was extremely flexible and provided for a short response time. It 
was a wonderful tool in emergency situations and for use when trying to satisfy 
particularly difficult customers. Also, the JOC contractor was able, because of the 
constant flow of work and established presence in the area, to be cooperative with our 
ever-changing schedules. Quality varied from acceptable to outstanding, depending on 
the circumstances (and the customer’s opinion, which may or may not have a basis in 
construction evaluation). In some cases, JOC prices were higher than those normally 
obtained the local market, but the JOC contract was not intended to be the most 


economical means of building something; 1t was intended to be expedient and flexible. é 
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Our contractor literally built from specifications developed on a walkthrough and jotted 
down on someone’s tablet. They also built from fully developed plans. Frankly, we had 


fewer problems with the less developed plans. 


I think the JOC process is a wonderful way to direct business to the local 
community without the smaller firms having to deal with government red tape. It isa 
shame that when 90% of the JOC business ends up on the hands of small businesses that 


this is not allowed to be counted towards the Navy’s Small Business Goals.” 


“Works well at our site. We have a top-notch contractor, [contractor]. They have 
JOC contracts around the country. They know what they are doing! They’re in it for the 
long haul, not short term profit. They have never filed a clam. We’ have very open 
communication. Our trust level is extremely high. But - they’re not perfect. They make 


mistakes. Some task orders should be done quicker. 


It’s a great contracting tool!!!” 


“As a member of the Small Projects Team here we have various innovative 
contracting vehicles at our disposal. JOC is one of those vehicles. It works very well in 


some cases but there are cases where it is not the proper vehicle.” 


“1. Provide expeditious process in awarding construction contracts. 
2. Joint scoping process which leads to less or no modification. 


3. Best way to obtain contracts for non-repetitive or one of a kind projects due to multi- 


discipline capability of the contractor. 


4. Price negotiation made easy due to usage of unit price book line items and use of PD3 


estimating software.” 


a 


“I have been involved with JOC going on 11 years. The lesson that I have learned 
is to have a good relationship with the contractor and have a contractor with a good 
track record. My advice if I was selecting a JOC/SABER contractor is don’t always look 
at the low bid. The worst JOC contract you can have is one that the contractor is losing 
money. When this happens all performance in construction, inspection, and staffing falls 
below average. Do some calling, check other places where the contractor has worked, 
and ask about their performance. This could save a lot of problems later on. Plus, know 


your area and what coefficient will work.” 


“Contract concept works very well. Very important to meet (partner) with 


contractor management weekly.” 


“{Contractor] has been at this [facility] for 10 years and they have done a fine ‘ 
job. They have a healthy attitude toward their tasks and generally bend backwards to | 


help please the customers. Good contractor!” 


“#1 problem here at [facility] the contractor bid a negative coefficient. The 
project negotiation started off with contractor fighting a 29% national average drop. 


Their means to make this up 1s added line items which slow down the SABER process.” 


“Contractor is concerned about customer service, which is the key to American 
business. They are not greedy & do not throw problems or comments back on the 


government.” 


“Regional application of the JOC concept is sometime difficult; however, it gives 


our agency flexibility in contracting.” ’ 


75 


“At [facility], we have implemented a new concept for Army installations - 
‘Incidental Design’. For those projects that require design, a two-step process is 


introduced: 


1. RFD: Contractor prepares and designs project and submits to government for 


approval. 
2. RFP: Once design is approved, RFP is issued, leading to a delivery order. 


We have been testing this system for one year now.” 


“The SABER contractor is fairly new, but (contractor name deleted) has 
delivered quality jobs up to this point.” 


Contractor comments: 


“1. Continuous building of TRUST between contractor/owner/customer is 





crucial. Give & take is necessary to make the contract work. If foundation of trust is 


low, give & take is lost and battles are eminent. 


2. Partnering is important. ‘Partnering is not a contract; It is a moral commitment 
for cooperation founded on win/win relationships. It is a strategy that recognizes the 


value of personal relationships based on good faith and trust.’-Simpson Hayward, Inc.” 


“The University is an honest and straightforward owner. It is these two qualities 
that make this a workable relationship. If they were not open and trustworthy, we would 


not be able to respond to their need as promptly, effectively, and efficiently.” 


“This was a very good project for us. Getting used to the U.P.B. was a learning 


experience for all. Our projects would range from $2500.00 to over $300,000.00. The 
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submittal process from our subcontractors was always a slow process. The subcontractor 
base around [facility] was not very large. This would cause problems in getting the 
projects completed in a timely manner. I believe our working relationship with the client 


was very good.” 


“Some tasks are difficult strictly by nature of the contract, 1.e. [facility] is difficult 
because of the ever changing workload and others, 1.e. [facility] are difficult because of a 
lack of knowledge in the JOC concept. Also, we have found that the best way to keep 


the process on track is through Partnering sessions.” 


“Current contract is a continuation of previous contract. Although the admin. 
effort is higher than the previous contract, we feel that we are less likely to have 
extensive or drawn out negotiations due to the experience level of both contractor and ‘ 


government personnel.” 


‘High levels of trust and great partnership help to accomplish our common goals. 
We have to strive to make JOC easier to use as 1t appears cumbersome on the front end. 
We have to apply and offer our expertise to add value to the JOC process. It is very 
responsive. On many occasions, quick turnarounds are only possible through JOC. A 
great tool at year end to commit funds. Easy to use to put projects ‘on the shelf’. One 
year later, only coefficient needs to be adjusted to award. More design authority would 


provide greater flexibility for government to use as tool.” 


“The reason that the JOC has been so successful here 1s because of the trust that 
has been developed between the contractor, Corps of Engineers, and the owner. We 
have been here for 8 years working with the same people day in & day out and we have $ 


become a team.” 


77 


“Current contract is up for rebid. We are actively pursuing another contract term, 
and are in the process of negotiation now. The government and our company both want 
to continue our contract. Hopefully, all will work out for the best. This has been a 


successful contract for all parties.” 


“We have a very good working relationship with the Army and COE. | attribute 
our continued success (2 JOC projects) to stability of the [contractor] team, our 


knowledge of construction, and our close relationships with the owner.” 


“The process of construction demands consistency in expectation by the 
contractor from the owner (gov). Probably the greatest deterrent to a successful 
Partnering is how each SABER contracting officer interprets their duty requirements of 
the contractor. With each new change of personnel, a new way of doing things is 
implemented, some better than others but a moving target at best. It is important to 
establish a contracting arm of SABER, preferably civilian, that executes the contractual 
obligations and expectations in a consistent manner. The other problem with SABER 
rests in competency of the CE overseeing the work. In many cases, undereducated CE 
are given the task to make decisions affecting project performance of the Contractor and 
have the power to prepare performance evaluations. In some cases, evaluations are made 
by CE not even on base or in the United States when the work was performed. These 
negative evaluations affect interaction of contractor & CE/SABER, diminishing potential 


benefits of Partnering process.” 


“We have had to work hard at this contract, probably because of the number of 
people involved. We are working with 4 Corps of Engineer districts in 5 different states. 
Negotiations are held with quite a large number of participants. Inspections, as you can 


imagine, can be trying. We are working with subcontractors that we haven’t been 
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comfortable with. It is getting a lot better as we get further along in our contract. The 
districts are getting to know us and the way we work. I do believe that I could probably 


answer these questions higher in another year.” 


“The gov’t seems to have a problem following the procedures that they have 


established.” 


‘This is our third 5-year contract at [facility]. We feel we have a good rapport 
with our SABER counterparts and the individual users on base. Our name is recognized 
and we have a reputation for delivering a quality product for a reasonable cost for sooner 


than the traditional method of ‘street bidding’.” 


* 
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Non-Partnered and Source Selection Contracts 


Owner comments: 


‘Since this is our first year with a JOC contract, there are growing pains. The 
contractor has come in with a limited staff expecting a fairly high amount of 
construction. I feel that the contractor could have played a bigger role in “selling” 
themselves as well as the JOC concept. In the interest of a long term relationship, they 
should be exceeding expectations in order to “win” over. users. Unfortunately, [owner] 
has a competitive renovations group (in-house) that makes the environment less clear 
cut. Since customers can go either way, they tend to seek pricing to see which alternative 
is better. We are also trying to establish clear cut procedures for administering JOC 
projects. Once these are more entrenched and users become comfortable, the contract 
should be more effective. An unsophisticated owner’s knowledge of proper JOC 


structure (owner resources and procedures) has caused us a slow start. 


“The JOC process is a good one if the contractor takes a serious interest in its 
success. We have been exposed to about four project managers during the 9 months of 
this contract and at times we were unsure of who was truly in charge. They have never 
provided us with sufficient quality control, causing the government to reject a lot of the 
installed work and have them re-do. They used a lot of unqualified sub-contractors to 


perform tasks.” 


“Although the negotiated construction costs may be slightly higher than 
competitive bidding, when the total acquisition costs are considered, JOC is a great 
bargain. If there is a drawback on the admin. side, JOC requires constant communication 
and close monitoring since we lack a formal set of drawings and specifications. The 
technical specs that are incorporated into the contract are so generic they have little 


impact. Questions regarding materials, methods, scheduling, field adjustments, and 
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customer-generated requests for changes must constantly be addressed. 


“Currently we are using a UPB (unit price book) that is approx. 6 years old. For 
the next JOC contract I will go with R.S. Means as the price book. JOC is still the best 
bang for the buck as far as contracts go in the U.S. Army. Now with the change in 


AFARS, our job is getting easier all the time.” 


“The Air Force utilizes the JOC contractor through the Corps of Engineers. 
There is usually a large communication gap from the Air Force’s requests to the 
contractor’s delivery. It seems that scheduling problems, site coordination problems, and 


customer dissatisfaction is the norm.” 


Contractor comments: 7 


“Negotiations are not structured to evaluate the appropriate line item & quantity 
required as intended. The owner’s contract & tech. representatives consistently fixate on 
the price of the item. Another practice that I find highly irregular (other contracting 
officers have identified it to be illegal considering this is not a procurements contract) is 
the technical rep’s tendency to solicit bids from other subs and suppliers as a bargaining 
tool or point of negotiation. They have even gone so far as to omit the line items from 
negotiation that the contractor will realize a profit and overhead gain, and then to retain 
those line items that they know the contractor will have to furnish or perform the labor 
and materials at cost or a loss. This is an unusual practice that I have not had the 
opportunity to experience throughout the seven different JOC/SABER contracts that I 
have been involved with. It seems to me that there is not an adequate level of contract 


administration on the government’s side to maintain the checks and balances.” 
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“Owner is very understaffed, slow to respond, poorly coordinated - sets high 


goals for contractor but does not provide necessary support - a very irritating customer.” 


“JOC process at [facility] is very good overall. Most people involved are fair and 


reasonable.” 


[Owner] has the most frequent turnover of both contracting & CE staff and the 
vast majority are trainees. This applies mostly to contracting. Contracting does whatever 


CE tells them to do. Unfortunately, Contracting takes the line of least resistance.” 


“Our policy does not include pursuing claims or disputes to any great length. I 
feel that this results in a rather one-sided contract administration on the part of our 
client. We are advocates of the Partnering process but have not been able to convince the 


owner to take the time or effort.” 


“Although there is no formal Partnering process utilized on this project, there is a 
strong ‘Team’ working relationship with our government counterparts. Building and 


maintaining trust is the key to the success of a JOC.” 
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Partnered and Low Bid Contracts 


Owner comments: 


“Limited to $100,000 per job order. Submittal process needs some improvement. 
Provides good flexibility with having to ‘rebid’ projects to eliminate items to get within 


budget.” 


(No contractor comments received.) 
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Non-Partnered and Low Bid Contracts 


Owner comments: 


“Engineering requires full drawings from the contractor, which is not really what 
SABER is designed for. Therefore it takes the contractor more than 30 days to design 


some complex projects.” 


“I don’t approve of using the Means as a cost base - too pricey and inflated. I’d 


much prefer to use negotiation of real costs. It’s less argumentative.” 


“Disregarding federal contracts, one of the issues for state agencies to contend 
with is the contracting restrictions within their state statutes. If unit pricing is stated to be 
acceptable by state statutes, then JOC contracting can be done. Otherwise, the state 
agency will have to ‘politic’ the state legislature to enact changes in the statutes to allow 
unit pricing. We were lucky in Oklahoma with our statutes. A second comment is that 
both sides, owner and contractor, have to work hard to develop a sense of both being on 
the same ‘team’. A sincere spirit of teamwork wins, an adversarial spirit loses. This is the 


single most important factor, in my mind, to having a successful JOC contract.” 


“The biggest problem we have at this time with our contract is the estimates, in 


particular with the mech., elect., and plumb. portion.” 


“This is the best IDIQ contract of the 3 I have worked with for the last 15 
months. This was the only one awarded with competition. The other 2 were 8(a)s. The 
IDIQ 1s an efficient way for us to do multi-trade construction. We have 10 people in 


contracts here with a budget of approx. $14M. The IDIQ desk did $5M (more or less).” 


an 


“Good contractor allows for consistently well done projects & follow up work. 


Exceeds usual performance of IFB contractors due to on-going relationship.” 


“This contractor is good to work with. Changes/modifications are easily 


negotiated and timely. Quality of work: we are getting what we are paying for.” 


“J wasn’t sure how to answer #21. Compared to other delivery methods, JOC is 
efficient in terms of quick execution and obligating year-end money. If ‘value-for-the- 
dollar’ is the measuring stick, JOC is slightly worse than other delivery methods in 
regard to efficiency. Overall, the 7-year experience with JOC on this base was positive. 
The first five years we were blessed with a good contractor. Their 17.29% coefficient 
allowed them to bend when we needed them to and still realize a profit. The aetihictel 
coefficient of the last two years bought us unproven subcontractors, modifications, a 

&- 


claims, and user dissatisfaction.” 


Contractor comments: 


“The greatest problem that I witness with this particular program is the lack of 
representation on the contracting side. Inexperienced or seemingly unqualified persons 
have held the position of contracting officer for the past 1.5 years. They have based their 
contractual determinations, on a large part, from opinions or advisement from the civil 
engineering section. This makes the contract administrative efforts seem biased and 
hence frustrating at best. A better understanding of the FAR as it pertains to 
SABER/JOC (in this case SABER) would seem to be helpful to the administration of 
these types of contracts. They occasionally make rulings as if the contracts were lump- 
sum (street or IFB contracts) based on their past contractual experience, which is not the 
case with the SABER programs, as I understand it. This opinion is based on my 
involvement with the SABER program at [facility], one of the first group of SABER e 
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contracts awarded & and the most completely trained contracting groups that I have 


encountered.” 


“The University inspectors are inexperienced with this type of contract. Very 
poor estimators which makes negotiations difficult. University politics makes life 
interesting. The relationship between traditional A&E and the shops and the JOC 
program needs work. The design is out of the contractor’s hands which slows the 
process down. The client and the contractor could both benefit from a longer base 
contract period with multiple option years. The overall relationship between Physical 


Plant and its clients is not pleasant. JOC is changing that!!”’ 


“Physical Plant has too large a workforce so by their own admission we only do 
jobs they don’t want to do or can’t get to, also customers they can’t get along with. I 
still like the contract and we hope to turn the tables on Physical Plant where we will get 


more work and better jobs. Trust only comes with performance.” 


“This contract was pre-empted by the Task Order Contract this year. Due to the 


fine line on our margin, my company opted not to bid on the Task Order Contract.” 
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